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Letter to the Editor 



To the Editor, 

Thank you for the opportunity to respond to "The CRE- 
ATE Method Does Not Result in Greater Gains in Critical 
Thinking than a More Traditional Method of Analyzing the 
Primary Literature" by M. Segura-Totten and N. Dalman 
(JMBE 14: 166-175, 2013). With regard to the comparison 
of the CREATE strategy and a "more traditional" approach, 
our comments focus on the nature of the "traditional" 
teaching experienced by the non-CREATE group, the degree 
to which the CREATE strategy was actually used, and the 
critical thinking assessments. 

The study compares a modified version of CREATE with 
"traditional" teaching with primary literature. The "tradi- 
tional" teaching described in this study does not match our 
experiences. We view "traditional" to mean that an instruc- 
tor assigns complete journal articles to students, with the 
intent that students will read (with no guidance) in prepara- 
tion for discussion. Subsequent class "discussions" typically 
involve either the faculty member lecturing about the paper 
or individual students presenting it, lecture-style. In this 
format, discussion usually is limited as only a small subset of 
students engage and most of the dialogue occurs between 
the instructor and the presenting students. We experienced 
such teaching ourselves, both as undergraduates and gradu- 
ate students. Indeed, in our early years of college teaching, 
we used such approaches with primary literature. Faculty 
trained in our recent summer workshops also report that 
this is a typical format for teaching with primary literature. 
Our lack of success with this traditional approach was part 
of what motivated us to develop CREATE. 

I n contrast to this view, the "traditional class" described 
by Segura-Totten and Dalman is much more directed and 
models many features of the CREATE strategy. Some aspects 
typical of CREATE teaching, for example initially distributing 
the papers without their titles and abstracts (to us a "non- 
traditional" approach we have not seen used in other con- 
texts), were used by the authors in both the "CREATE" and 
"traditional" student cohorts. Pre-class preparation in the 
"traditional" group involved written homework addressing 
some questions that mirror the pre-class preparation with 
CREATE tools. The significant overlap in the two teaching 
approaches used in this study makes comparisons difficult. 

We consider the version of CREATE used in the study 
to be a limited version of the strategy we have described 
(1-3). In our version, students read a series of papers from 
one lab group or from different groups pursuing the same 
scientific issue; thus students see the evolution of a research 
trajectory over time. Students carry out multiple experi- 
mental design/grant panel sessions during their CREATE 



semester. This repeated practice builds their prowess at 
both designing and intelligently criticizing experiments. Our 
CREATE students "build their own textbooks" by compiling 
notebooks/portfolios that include their annotated articles, 
concept maps, cartoons, notes from in-class activities, and 
additional information they have gathered on their own to 
support their understanding. The portfolios are resources 
for open book assessments we give during the semester. 
Much class time focuses on analyzing "how the experiment 
was done," with student cartoons demystifying methodol- 
ogy. Students use data templates to pull together their 
insights from pre-class cartooning and annotation and draw 
conclusions about data from individual experiments, ideally 
interpreting the data as if they had performed the studies 
themselves. Our CREATE students also carry out email 
surveys of paper authors as an inroad to understanding the 
motivations of working researchers; authors' candid and 
thoughtful responses have strongly influenced students' 
sense of both scientists and of the research process (see 
Tables I, SI in reference 2). Many of these features of CRE- 
ATE were lacking or substantially modified in the version 
of CREATE used in the Segura-Totten and Dalman paper. 
In addition, it is not clear how much discussion time was 
devoted to "how" experiments were done or "why" ex- 
periments were done in particular ways (such discussions 
get into roles of controls, experimental design and the 
like, and are standard features of our CREATE courses). 
In our CREATE classrooms, lecture is rare or absent, and 
discussions do not move directly from defining hypotheses 
to discussing results. 

Finally, the critical thinking assessments reported in this 
paper are very different from those we have used to address 
the same issue. We have used two assessments, a set of 
questions derived from the Field Tested Learning Assess- 
ments (www.flaguide.org); and the Tennessee Tech Critical 
Thinking Assessment Test (CAT) (5) to document significant 
pre-course/post-course gains in CREATE students. Neither 
test is distributed in advance, and both require reasoning 
and analysis in topic areas separate from those of our CRE- 
ATE classes; thus we think it likely that each test measures 
transferable thinking skills. In the Segura-Totten and Dal- 
man paper, critical thinking was assessed in two ways. One 
involved analyses of journal articles, portions of which were 
distributed a week in advance of the in-class assessment. A 
second assessment focused on student responses to indi- 
vidual exam questions designed at particular Bloom levels. 
Whether these assays and ours address the same aspects 
of critical thinking is an open question. 

We look forward to studies that explore how CREATE 
compares with other methods of teaching and learning from 
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primary literature. We suggest that the use of CREATE 
portfolios, open-book testing, repeated experimental design/ 
grant panel activities, and email surveys of authors be included 
in studies of the CREATE strategy. We see these features, 
coupled with intensive discussion of how experiments were 
carried out and what the reported data mean — complement- 
ed by sustained focus on the development of a project over 
time — , as contributing significantly to the range of cognitive 
and affective gains, including critical thinking gains, we have 
documented in CREATE students (I, 2, 4). 

Sincerely, 

Sally G. Hoskins, Ph.D. 

Biology Department 
City College of the City University of New York 

New York, NY 
www.teachcreate.org 

Kristy L. Kenyon, Ph.D. 

Biology Department 
Hobart and William Smith Colleges 
Geneva, NY 
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